When selection acts on social or behavioral traits, the fitness of an individual depends on the phenotypes of its competitors. Here, we describe methods and statistical inference for measuring natural selection in small social groups. We measured selection on throat color alleles that arises from microgeographic variation in allele frequency at natal sites of side–blotched lizards (Uta stansburiana). Previous game-theoretic analysis indicates that two color morphs of female side–blotched lizards are engaged in an offspring quantity–quality game that promotes a density- and frequency–dependent cycle. Orange–throated females are r-strategists. They lay large clutches of small progeny, which have poor survival at high density, but good survival at low density. In contrast, yellow–throated females are K-strategists. They lay small clutches of large progeny, which have good survival at high density. We tested three predictions of the female game: (1) orange progeny should have a fitness advantage at low density; (2) correlational selection acts to couple color alleles and progeny size; and (3) this correlational selection arises from frequency-dependent selection in which large hatchling size confers an advantage, but only when yellow alleles are rare. We also confirmed the heritability of color, and therefore its genetic basis, by producing progeny from controlled matings. A parsimonious cause of the high heritability is that three alleles (o, b, y) segregate as one genetic factor. We review the physiology of color formation to explain the possible genetic architecture of the throat color trait. Heritability of color was nearly additive in our breeding study, allowing us to compute a genotypic value for each individual and thus predict the frequency of progeny alleles released on 116 plots. Rather than study the fitness of individual progeny, we studied how the fitness of their color alleles varied with allele frequency on plots. We confirmed prediction 1: When orange alleles are present in female progeny, they have higher fitness at low density when compared to other alleles. Even though the difference in egg size of the female morphs was small (0.02 g), it led to knife–edged survival effects for their progeny depending on local social context. Selection on hatchling survival was not only dependent on color alleles, but on a fitness interaction between color alleles and hatchling size, which confirmed prediction 2. Sire effects, which are not confounded by maternal phenotype, allowed us to resolve the frequency dependence of correlational selection on egg size and color alleles and thereby confirmed prediction 3. Selection favored large size when yellow sire alleles were rare, but small size when they were common. Correlational selection promotes the formation of a self-reinforcing genetic correlation between the morphs and life-history variation, which causes selection in the next density and frequency cycle to be exacerbated. We discuss general conditions for the evolution of self-reinforcing genetic correlations that arise from social selection associated with frequency–dependent sexual and natural selection.

Corresponding Editor: J. Losos